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Abstract Body 


Background / Context: 

Relatively few rigorous studies of the effeetiveness of eontemporary publie prekindergarten 
programs have been eondueted despite the growing number of programs and large monetary 
investments that aeeompany them. The study on whieh this presentation is based was launehed in 
partnership with the Tennessee State Department of Edueation’s Division of Sehool Readiness 
and Early Eeaming to provide an assessment of the effects of the statewide Tennessee Voluntary 
Pre-Kindergarten (TN-VPK) program on the readiness for kindergarten of the economically 
disadvantaged population it serves. TN-VPK has become a controversial program in Tennessee, 
with some legislators expressing doubts about its value in the context of severe budget shortfalls 
and still others referring to it even more skeptically as expensive babysitting. This study 
interleaves a randomized control trial (RCT) design and an age-cutoff regression discontinuity 
(RD) design to evaluate the effectiveness of the TN-VPK program. Though the project is still 
underway, this presentation will summarize results from the RCT through first grade. 

Purpose / Objective / Research Question / Focus of Study: 

This study includes a randomized control experiment, but that component could only be 
implemented in a limited number of schools with more applicants than seats in the pre-k 
program. Using the RCT, this study seeks to determine the statewide effect of TN-VPK on both 
the kindergarten readiness of the participating children and their long-term cognitive and non- 
cognitive skills. 

Setting: 

TN-VPK supports 934 pre-k classrooms that serve more than 18,000 at-risk children across all 
95 Tennessee counties, in 133 of the 136 Tennessee school systems, though all counties/systems 
are not represented in the sample for this paper. It is truly a statewide program serving all areas 
of the state, both rural and urban. 

Population / Participants / Subjects: 

To be eligible for TN-VPK, children must be age four on or before September 30 of the 
respective school year. By statutory requirement, the program gives top priority to children who 
qualify for the Tree or Reduced Price Eunch Program, and 86% of the children enrolled 
statewide meet that criterion. The full randomized sample of 3,025 students in this study was 
26% Black, 22% Hispanic, 50% male, and 76% Native English speakers. The consented 
subsample of 1,076 students was 23% Black, 18% Hispanic, 45% male, and 80% Native English 
speakers. There were two cohorts of children, one that applied to TN-VPK in 2009 and another 
that applied in 2010. 

Intervention / Program / Practice: 

TN-VPK operates through competitive grants to local school systems who apply for approval 
and funding. Those grants support only a portion of the necessary classroom funding, the balance 
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must come from other sourees. This arrangement permits and eneourages eollaboration between 
sehool systems and other organizations. In this “eollaboration model,” sehool distriets may 
operate their pre-k programs through eollaborative agreements with loeal nonprofit and for-profit 
ehild eare providers and Head Start programs so long as those ageneies have attained the highest 
rating from the lieensing system administered by the Tennessee Department of Human Serviees 
and meet the TN-VPK standards. Those standards, whieh are set by the State Board of 
Edueation, require the following; 

• A state lieensed teaeher with an early ehildhood edueation endorsement in eaeh 
elassroom; 

• A teaeher assistant who holds or is aetively working toward at least a CDA or assoeiate 
degree in early ehildhood; 

• Professional development support for teaehers; 

• An adult-student ratio no smaller than 1:10; 

• A small elass size maximum of 20; 

• An approved age-appropriate eurrieulum aligned with the Tennessee Early Childhood 
Edueation Developmental Standards; 

• A family engagement eomponent and a pre-k to kindergarten transition plan for eaeh 
ehild; 

• Vision, hearing, and health sereening and referral serviees; 

• A minimum of 5.5 hours per day, exelusive of nap time, for a minimum of 180 days per 
year within a ealendar that ineludes 200 working days of 7.5 hours for teaehing staff. 

Research Design: 

Eor the RCT portion of the study, in the summer before eaeh of two sehool years, administrators 
in sehools that expeeted more applieants to their TN-VPK program than available slots were 
asked to partieipate in the RCT. All eligible applieants were plaeed on a list that was randomized 
by the researeh team. Children were offered seats in the VPK program in the order that they 
appeared on the randomized list. This proeedure gave eaeh ehild an equal ehanee to be ranked 
high enough on the list to be admitted but, also, by the same equal ehanee, left some ehildren too 
low on the list for a seat to be available for them. The total sample aeross both eohorts ineluded 
more than 3,000 ehildren, and those ehildren are being traeked through the state database. 
Outeomes sueh as grade retention, diseiplinary aetion, sehool attendanee, and speeial edueation 
plaeement are eolleeted from the database eaeh year. A subsample of over 1 ,000 of these 
randomized ehildren was individually eonsented and assessed at the beginning and end of their 
VPK year, at the end of kindergarten and first grade, and will be assessed at the end of seeond 
and third grade, as well. 

Data Collection and Analysis: 

Children in the eonsented substudy are individually assessed using a set of Woodeoek Johnson 
III aehievement tests of pre-reading, language, and mathematie skills (Eetter-Word 
Identifieation, Spelling, Understanding Direetions, Applied Problems, Quantitative Coneepts, 
Passage Comprehension, and Oral Comprehension). Teaeher ratings of ehildren’s eognitive, 
soeial, and behavioral skills were also eolleeted. All analyses were done in three-level 
regressions with ehildren nested in their pre-k elassrooms and distriets, regardless of the timing 


SREE Spring 2014 Conference Abstract Template 


2 



of the outcome variable. Propensity scores were created to reduce baseline differences between 
the two experimental conditions in the consented subsample, and those propensity scores as well 
as the variables that went into their creation were included in the analytic models testing main 
effects. Analytic models for non-cognitive outcomes were slightly different depending on the 
nature of the outcome and the sample used. At the time of this writing, RCT data are available 
for the both cohorts of children in the consented intensive subsample through first grade. 

Findings / Results: 

Though data collection is continuing in both components of this study, a large portion of 
outcome data for the RCT portion has been analyzed already. The results of the RCT so far 
indicate positive and significant effects at kindergarten entry for children who had participated in 
TN-VPK when compared to children who did not attend TN-VPK on all of the directly assessed 
achievement scales examined, with effect sizes ranging from .10 to .46. The analogous 
improvements for the program participants relative to the non-participants ranged from 21% to 
89%. Additionally, there were statistically significant differences favoring the children who had 
attended TN-VPK on the kindergarten teachers’ ratings of kindergarten readiness and work- 
related skills, obtained at the beginning of the kindergarten year. There were no differences on 
the other kindergarten teacher rated measures. However, differences between the treatment and 
control groups were greatly diminished by the end of kindergarten and were no longer 
significant. Similarly, group differences at the end of first grade on the directly-assessed 
academic measures were not significant, with one exception for which non-participants had 
significantly higher scores than TN-VPK participants. There were no differences on teacher 
rated measures at the end of first grade. Table B1 in Appendix B provides effect sizes for group 
differences across Woodcock Johnson subscales at each outcome timepoint. 

Despite the lack of intervention effects after the pre-k year, we are beginning to see suggestions 
of effects on non-cognitive outcomes for program participants. In the consented subsample, TN- 
VPK participants were retained in kindergarten significantly less often than non-participants; 
4.1% of participants were retained compared to 6.2% of nonparticipants. This effect was 
replicated in the portion of the full randomized sample for whom data were available; 4.0% of 
participants were retained in kindergarten in that sample relative to 8.0% of the nonparticipants. 
Additionally, in the full randomized sample, attendance in kindergarten and first grade was 
significantly greater for TN-VPK participants than nonparticipants by close to 2 days in 
kindergarten and 3.5 days in first grade. 

Conclusions: 

As is likely the case with other scale up state-supported prekindergarten programs, there is no 
consensus in Tennessee about what TN-VPK is expected to accomplish. If the expectation is that 
the economically disadvantaged children who attend TN-VPK will enter kindergarten better 
prepared, this research shows that TN-VPK fulfills that expectation. TN-VPK children gained 
more on early achievement measures during the pre-k year than children who did not attend and 
were rated as better prepared by their kindergarten teachers. If the expectation is that one year of 
TN-VPK will make these children perform better throughout their school careers, it is too early 
to tell how effective the program is. Though the early achievement measures show no sustained 
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differences between attenders and the control children, we do not yet know how these children 
will perform on the state achievement tests in third grade or the long-term effects on other 
aspects of school performance, early indications of which have already appeared. 
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Appendix B, Tables and Figures 


Table B1 

The Effects of TN- VP K Participation on Woodcock Johnson Subscales across Time 


EndofPre-K End of K End of 1st Grade 


Subscale 

Effect Size 

p-value 

Effect Size 

p-value 

Effect Size 

p-value 

Eiteracy 

Eetter-Word Identification 

.46 

.000 

.04 

.522 

-.06 

.399 

Spelling 

.25” 

.000 

.01 

.922 

-.09 

.143 

Eanguage 

Oral Comprehension 

.10 

.036 

.09 

.202 

-.07 

.257 

Picture Vocabulary 

.20” 

.000 

.08 

.179 

.02 

.688 

Math 

Applied Problems 

.14” 

.001 

.02 

.727 

-.05 

.346 

Quantitative Concepts 

.31” 

.000 

-.07 

.187 

-.21” 

.004 


** p <.01 
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